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CONTROLLING COMMUNICATIONS THROUGH A VIRTUAL REALITY 

ENVIRONMENT 



FIELD OF THE INVENTION 

The present invention relates to virtual reality 
systems, and particularly relates to using virtual 
reality systems to facilitate intuitive communications 
through communications systems associated with a virtual 
reality environment . 



BACKGROUND OF THE INVENTION 

Advances in communications technology continually 
expand and refine available options for communicating 
with one another. Key examples come to mind, such as the 
proliferation of wireless communications devices, audio 
and video conferencing systems, network and Internet data 
sharing systems, and electronic collaborative work 
applications. Ironically, exploiting new communications 
and information technology becomes increasingly 
complicated as the range and sophistication of the 
underlying technologies increases. 

Workers in a typical company communicate with one 
another in a variety of ways. For example, a single 
person may be associated with a desk phone, a cellular 
phone, a pager, a personal digital assistant, a computer 
system, multiple email accounts, and a myriad of other 
points of contact. The preferred means of contacting 
such a person will likely vary with changing 
circumstances— such as time of day or travel status. The 
efficiency of co-workers attempting to contact this 
person may be compromised by the sheer number of contact 
choices available, the number of systems involved (e.g., 
phone, email, paging) , and by not knowing which choice is 
appropriate at any given time. 
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Other barriers to efficient and effective 
communications stem from the increasing complexity of the 
information involved in our communications. Workers are 
involved in the development of increasingly complex 
systems that are oftentimes developed using 
geographically disperse, multi-disciplinary teams. The 
ability of these teams to interact with one another and 
share information intuitively and effectively is critical 
for success. Further, consumers buy and use increasingly 
sophisticated systems and services, yet their ability to 
realize the full value of such systems and services 
depends on a vendor's ability to supply them with 
appropriate, readily understood assistance and supporting 
information . 

Unfortunately, existing communication solutions 
remain loosely integrated at best and do not offer users 
with an intuitive way to interact with and use any and 
all of the communications technologies available in a 
given situation. Accordingly, there remains a need for a 
system that combines or has access to disparate 
communications technologies and provides an intuitive 
interface, preferably visual, that allows people to 
intuitively interact with other persons and with 
supporting information and communications systems. 



SUMMARY OF THE INVENTION 

People intuitively understand how to work within a 
graphical computer application in which desired functions 
are represented in a meaningful manner. The present 
invention simplifies communications between people and 
systems by linking specific communications functions to 
events occurring within a virtual reality environment. 
The virtual reality environment provides a graphical 
representation of places and objects that can be 
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associated with real-world persons and communications 
systems. For example, a telephone call might be 
automatically initiated between two co-workers based on 
one of the co-workers virtually entering the virtual 
office of the other. 

More generally, in the present invention, a virtual 
reality system defines a virtual environment and events 
that can occur within the virtual environment, and 
initiates desired real world actions when an event 
occurs. System users, represented as avatars within the 
virtual environment, generate virtual events based on 
interacting with virtual objects and locations, as well 
as with other avatars. In general, virtual entities 
(e.g., avatars, objects, and locations) are associated 
with data bearing on the particular real world action 
triggered in response to an event involving the entity. 
For example, an avatar entering a virtual office 
(location) might generate a virtual event that triggers a 
telephone call or other electronic communications between 
a person associated with the avatar and a person 
associated with the virtual office. The associated data 
in this case could include the telephone numbers or 
contact information required to establish communications 
between the persons involved. Other data might specify a 
computer application to open facilitating data transfer 
or file sharing activities. 

A given virtual event may have a single predefined 
real world action. For example, one avatar handing a 
virtual file folder to another avatar might invoke a file 
sharing application, or simply transfer associated data 
from one computer system to another. Alternatively, a 
given event may trigger different real world actions at 
different times. Thus, entering a virtual office during 
normal business hours may initiate a phone call to an 
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office telephone, while the same event might trigger a 
call to an after-hours number, enter voice mail, or 
launch an email application, if the same event occurs 
outside of normal business hours. 

Multiple event types and multiple possible real 
world actions associated with each event type may be 
defined and configured for each virtual entity- 
Normally, an owner of a given virtual entity performs 
such configurations. This allows a person to configure 
the real world actions that are initiated when one of 
their virtual entities is interacted with in the virtual 
environment. This allows the person to choose how they 
wish to be contacted at different times, or in different 
locations. Additionally, the virtual reality system 
itself may use changing conditions, such as time of day 
or the whereabouts of the person, to determine what real 
world action is initiated in response to a given virtual 
event . 

Virtual event types include object events such as 
drop, pick up, touch, and use. Location oriented event 
types include enter, leave, and move. A wide range of 
other event types such as share, open, and send, may be 
defined for file sharing and collaborative working. The 
present invention also supports push technologies 
allowing multiple users to share a graphical environment. 
Event types may be extended as needed to support a broad 
range of virtual activities. The virtual reality system 
includes interfacing support allowing it to interface, 
either directly or indirectly, with a wide variety of 
systems, such as telecommunications networks, computer 
and information networks, computer applications, and a 
wide range of other devices and systems. Thus, a broad 
array of devices, systems, and software may be made 
responsive to events occurring within the virtual 
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environment. This provides substantial flexibility and 
makes the system and methods of the present invention 
suitable for corporate communications and collaborative 
working, virtual e-commerce, and sophisticated customer 
support applications . 

Those skilled in the art will immediately appreciate 
the range applications to which the present invention may 
be applied. Additionally, other aspects and features of 
the present invention will become apparent to those 
ordinarily skilled in the art upon review of the 
following description of specific embodiments of the 
invention in conjunction with accompanying figures. 



BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 illustrates an exemplary environment in 
which the present invention may be advantageously 
practiced. 

Figure 2A illustrates an exemplary embodiment of the 
system of the present invention. 

Figure 2B illustrates an exemplary initiation of a 
telephony session between a first and second user in 
accordance with the present invention. 

Figure 3 illustrates an alternate exemplary 
embodiment of the system of the present invention wherein 
multiple communications systems are involved. 

Figure 4 illustrates an alternate exemplary 
embodiment of the system of the present invention wherein 
a single communications system is involved. 

Figure 5 illustrates the configurable nature of 
communications functions in one embodiment of the present 
invention . 

Figure 6 illustrates how a single virtual entity may 
be linked to a common communications function, such as a 
conference call, in the present invention. 
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Figure 7 provides exemplary, simplified program 
logic for initiating communications functions in response 
to certain events occurring within the virtual reality 
environment . 

Figure 8 provides additional details for certain 
aspects of the program logic of Figure 7. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Before delving into details regarding the present 
invention, it is useful to build a framework of 
discussion and identify certain terminology. Within a 
virtual reality environment, virtual entities may be used 
to represent real world entities. A real world entity 
can be almost anything with examples ranging from an 
actual person, a communications device or other system 
associated with' a person or persons, a computer network, 
or a software application. Virtual entities may appear 
as virtual persons (avatars), or may be virtual objects 
or virtual locations. For example, a virtual reality 
system user might create a "virtual office" accessible to 
other users of the virtual reality system. With the 
present invention, in many of its various embodiments, 
virtual entities generally represent or are associated 
with one or more persons, communications and information 
systems associated with one or more persons, or computer 
networks and computer applications. 

Referring to the virtual office example, the "owner" 
of the virtual office might place a virtual calendar, a 
virtual phone, and a virtual PC on a virtual desk within 
the office. Another user operating within the virtual 
environment, possibly as an avatar, might virtually touch 
the calendar to schedule a meeting with the office owner, 
pick up the phone to call the office owner, or place a 
document on the PC to transfer a file to the office 
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owner's real world PC. This theme can be extended with 
virtually touching a virtual notepad resulting in an 
email being sent to the office owner. The theme may also 
be altered somewhat in that the office owner may 
configure the virtual office such that a preferred 
communication is initiated whenever another user 
virtually enters the office itself. Depending upon the 
whereabouts of the office owner, the communication 
automatically initiated using email, a mobile telephone, 
pager, personal digital assistant, an office telephone 
call, or a home telephone. 

With the above context in mind, Figure 1 illustrates 
an overall framework 10 for practicing an exemplary 
embodiment of the present invention. The invention 
incorporates or operates in conjunction with a virtual 
reality system 20. The components and related aspects of 
the virtual reality system 20 are identified in the 
accompanying drawings using reference number in the 200' s 
for clarity. The virtual reality system 20 includes a 
virtual environment server 210 hosting a virtual reality 
environment 220 having virtual entities 222 and 224, and 
a virtual environment interface 240. While two virtual 
entities 222 and 224 are illustrated, the virtual reality 
environment 220 may have any number of virtual entities. 
The virtual reality system 20 interfaces, directly or 
indirectly, with a communications system 30. 

In the context of Figure 1, users 40 and 50 
represent persons able to interact with the 
communications system 30 and the virtual reality system 
20. More generally, the users 40 and 50 are considered 
as real world entities associated with the virtual 
entities 222 and 224. As will be shown in more detail 
later, the users 40 and 50 will typically have some type 
of computer to interact with the virtual reality system 
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20 and some type of communications device to interact 
with the communications system 30. 

The users 40 and 50 are associated with the virtual 
entities 222 and 224, respectively- More particularly, 
various communications devices (e.g., phones, pagers, 
computers, personal digital assistants etc.) associated 
with the users 40 and 50 are associated with the virtual 
entities 222 and 224, respectively. Thus, virtual 
interactions between the virtual entities 222 and 224 
under control of the users 40 and 50 may be used to 
trigger or initiate desired communications functions 
through the communications system 30 involving the users 
40 and 50. The communications system 30 may, for 
example, be an intra-office PBX, the public switched 
telephone network (PSTN) , a network-based or Internet 
telephony system, a computer application, a computer 
network, or other type of information system. 

One or more events are defined for the virtual 
reality environment 220. These events are generated by 
interactions between the virtual entities 222 and 224. 
For example, the user 40 may direct his or her 
corresponding virtual entity 222 to somehow interact 
(e.g., touch, enter, contact, select) the virtual entity 
224 associated with the user 50. This action generates a 
defined virtual event detected by the virtual environment 
interface 240. In response to the event, the virtual 
environment interface 240 initiates a desired 
communications function that is defined for or associated 
with the event. For example, the virtual reality system 
20 might be configured with telephone numbers 
corresponding to telephones of users 40 and 50 such that 
a telecommunications call is initiated between them in 
response to the virtual event. 
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The virtual reality system 20 may include one or 
more computer systems. For example, the virtual 
environment server 210 may be deployed as one computer 
system and the virtual environment interface 240 as 
another computer system. The virtual environment 
interface 240 may itself be separate from, for example, a 
network server (not shown) , or may be included as part of 
the network server. The software for hosting the virtual 
reality environment, configuring and responding to events 
within the virtual reality environment, and interfacing 
with other systems to initiate a desired communications 
function, may be included in a single computer system, 
spread across multiple computer systems, or any 
combination thereof . 

Figure 2A provides details for one embodiment of the 
present invention. The user 40 is associated with a PC 
40A and a communications device 40B, while the user 50 is 
associated with a PC 50A and a communications device 50B. 
Users 40 and 50 use PCs 40A and 40B, respectively, to 
interact with the virtual reality environment 220. While 
two users (40 and 50) are illustrated, the virtual 
reality system 20 supports interaction with any number of 
users. As shown, the virtual environment interface 240 
supports such interaction. Other configurations of the 
virtual reality system 20 may involve the PCs 40A and 50A 
interfacing directly with the virtual environment server 
210 to enter and interact with the virtual reality 
environment 220. In such cases, the PCs 40A and 50A may 
communicate directly with virtual environment server 210 
and the virtual environment interface 240. 

The user 40 may use PC 40A to cause the virtual 
entity 222 to interact with the virtual entity 224. This 
interaction triggers an event that has been defined in 
the virtual reality environment 220. The virtual 
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environment interface 240 receives notification of the 
event and, optionally, additional event information. 
Based on its configuration, the virtual environment 
interface 240 identifies a desired communications 
function associated with the event and initiates this 
desired function in the communications system 30. The 
desired communications function might be, for example, a 
communications session between the users 40 and 50 using 
their corresponding communications devices 40B and SOB, 
respectively. 

Figure 2A also highlights some of the substantial 
flexibility associated with the present invention. For 
example, the communications network 30 might be a 
computer network and the communications devices 40B and 
SOB might be computer telephony devices. In this case, 
the PCs 40A and 50A may be used to communicate with the 
virtual reality system 20, as well as the communications 
devices 40B and SOB. Additional communications devices 
(not shown) may be associated with one or both users 40 
and 50. Thus, the virtual reality system 20 may be 
configured with information about contacting the users 40 
and 50 in a variety of ways, such as phone numbers, email 
addresses, and information regarding preferred methods of 
contact for different times of the day. Preferably, such 
information can be dynamically updated by the users 40 
and 50 (and other users— not shown— of the virtual reality 
system 20) , or by the virtual reality system 20 to 
reflect changing preferences for contacting users 40 and 



Figure 2B illustrates a simplified, logical sequence 
of events for practicing one embodiment of the present 
invention. In Figure 2B, the virtual reality system 20 
provides a virtual environment interface 240 for each of 
the users 40 and 50. In such implementations, PCs 40A 



50. 
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and 50A corresponding to the users 40 and 50, 
respectively, may run the virtual environment interfaces 
240A and 240B, respectively, allowing the users 40 and 50 
to operate within the virtual reality environment 220, as 
well as processing virtual events. The communications 
devices 40B and SOB can be, for example, Universal Serial 
Bus (USB) phones, Telephony Applications Programming 
Interface (TAPI) compliant phones, or any number of other 
types of communication devices suitable for PC 
interfacing. Assume that the virtual entity 224 is 
configured as a virtual office for the user 50. Further, 
assume that the virtual entity 222 is configured as an 
avatar representing the user 40. 

Operating under control of the user 40, the virtual 
entity 222 "enters" the virtual entity 224. This 
interaction generates an "enter" event (STEP 1) . The 
event notification sent to the virtual environment 240A 
of the user 40 includes event information identifying the 
type of event and the virtual entities involved. In the 
scenario where the PC40A of the user 40 is hosting the 
virtual environment interface 240A, the PC 40A invites 
the associated communications device 40B to initiate a 
communications session (STEP 2) . If the communications 
device 40B is operative and available, it affirms the 
communications session initiation (STEP 3) . Then, the PC 
50A of the user 50 receives the invitation information, 
which it passes to the communications device SOB (STEP 
4) . This assumes that the PC 50A of user 50 is hosting 
the virtual environment interface 240B. If the 
communications device SOB is operative and available, it 
affirms the invitation and this is acknowledged for the 
user 40 (STEPS 5, 6, and 7). Affirmation of the 
acknowledgement is sent back to the PC 40A (STEP 8), 
where it is acknowledged (STEP 9) and a communications 
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path between the communications devices 40B and 50B is 
established (STEP 10) . While many methods exist for 
supporting the above telephone session initiation 
actions, the Session Interface protocol (SIP) represents 
a standardized protocol supporting such implementations. 

Figure 3 illustrates the flexibility of the present 
invention. Three virtual entities 222, 224, and 226 
within the virtual reality environment 220 represent the 
user 40. Thus, there is a one-to-one correspondence 
between virtual entities 222, 224, and 226 and' the 
various communications systems 30, 32, and 34 available 
for contacting the user 40. Each virtual entity 222, 
224, and 226 may be associated with a different 
communications system 30, 32, and 34, respectively. The 
different communications systems 30, 32, and 34 can be 
essentially any type of communications system, such as 
analog or digital telephone networks, cellular networks, 
computer or other information networks, etc. The system 
interfaces 42 collectively represent the communications 
devices or systems used by the user 40 to interact with 
the communications systems 30, 32, and 34. As such, the 
particular communications function initiated by the 
virtual environment interface 240 depends upon which of 
the virtual entities 222, 224, and 226 are involved in 
the virtual event. 

Figure 4 illustrates a many-to-one mapping wherein 
multiple virtual entities 222, 224, and 226 are all 
associated with the same communications system 30 for 
contacting the user 40. In this context, virtual 
interaction with any of the virtual entities 222, 224, 
and 226 may be configured to initiate a communications 
function in the communications system 30 via the virtual 
environment interface 240. The utility of this 
arrangement can be better understood by imagining a 
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virtual reality environment 220 where the user 40 has 
multiple representations within the virtual reality 
environment 220. For example, the virtual entity 222 
might be an avatar representation of the user 40, while 
the virtual entities 224 and 226 might represent a 
virtual office and a virtual telephone, respectively, 
both associated with the user 40. 

Figure 5 illustrates how user information and 
certain conditions, such as time of day, may be used to 
extend the utility of the present invention. Again, 
virtual entities 222 and 224 correspond to the users 40 
and 50, respectively. The users 40 and 50 can interact 
with the virtual reality system 20 to configure the 
communication function (s) they desire to be initiated by 
interactions with their corresponding virtual entities. 
For example, the particular communications function 
initiated by the virtual reality system 20 upon detection 
of a virtual event involving the virtual entity 224 may 
be based on configuration information, a user profile for 
example, provided by the user 50. 

Figure 5 further illustrates the ability of the 
virtual reality system 20 to initiate different 
communications functions or modify the information used 
to initiate a given communications function, based on 
monitoring certain conditions. Intuitively, one imagines 
that a preferred method of contacting a given individual 
might change at different times during the day. During 
normal working hours, a desk telephone may be preferred, 
but during lunch or after hours, a mobile telephone may 
be preferred. At still other times, emailing or paging 
might represent the best or perhaps only way of 
contacting the given person. 

In these and other scenarios, the virtual reality 
system 20 may be configured to consider one or more 
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criteria for evaluating one or more conditions to 
determine which communications function is preferred at 
any given time. While conditions are illustrated as 
being externally provided to the virtual environment 
interface 240, the virtual reality system 20 may 
determine one or more of the conditions used to select 
preferred communications functions for a given virtual 
event at a given time. System time and date represents 
the type of conditional data that is easily maintained by 
the virtual reality system 20, while monitored locations 
of workers within an office complex might represent the 
type of conditional information provided by external 
sensing or monitoring systems. 

Figure 6 illustrates a configuration alluded to 
earlier, wherein a communications system 30, such as a 
video conferencing system, is represented as a virtual 
entity 222 within the virtual reality environment 220. 
In this way, the virtual reality environment 220 may be 
constructed to provide an intuitive way for users 40, 50, 
and 60, to join in' a conference supported by the 
communications system 30. One technique is to configure 
the virtual entity 222 as a virtual conference room 
within the virtual reality environment 220. 
Configuration information would then be defined and 
stored so that the virtual reality system 20 is capable 
of initiating a connection between the users 40, 50, and 
60, and the communications system 30 when the individual 
users interact with the virtual entity 222. 

Thus, Figure 6 is just one example of the 
flexibility of the present invention in representing 
users, communication systems, communication devices, 
computers, and various other real world entities as 
virtual entities (222, etc.) within the virtual reality 
environment 220. The particular communications function 
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initiated by the virtual reality system 20 may then 
depend upon what specific kind of real world entity is 
represented by a given virtual entity. 

Figure 7 illustrates operating logic for one aspect 
of operation of the virtual reality system 20 in some 
embodiments of the present invention. Operation begins 
(block 710) with the virtual reality system 20 
registering one or more real world entities (people, 
systems, software, etc.) so that desired associations may 
later be made between the corresponding virtual entities 
and the real world entities (block 720) . Once the real 
world entities are registered, including the 
establishment of any user profiles and conditional 
criteria, the virtual reality system 20 monitors for 
virtual events (block 730) . In the absence of any 
virtual events, the virtual reality system 20 continues 
monitoring (block 730) . Upon occurrence of one or more 
defined events (block 740), the virtual reality system, 
if necessary, determines the event type or identifies the 
specific event (s), and processes any supporting event 
information (block 750) . 

Note that in some embodiments, the virtual reality 
system 20 may have only one type of event. In this case, 
the event notification sent from the virtual environment 
server 210 to the virtual environment interface 240 
identifies the virtual entities involved in the event. 
Other embodiments may have many types of events. Some 
embodiments may define the following events: 
Touch (entityl, entity 2), PickUp (entityl , entity2), 
Drop (entityl, entity2), Enter (entityl , entity2), 
Leave (entityl, entity2), Share ( entityl , entity2), and 
Push (entityl, entity2). Supporting data, much like 
functional arguments included with function calls in 
conventional programming languages, identifies the 
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particular entities involved in a given virtual event and 
information bearing on the type of event. This allows 
the virtual reality system 20 to undertake one or more 
specific communications functions based on the type of 
event and the particular virtual entities involved in the 
event. Of course, the present invention is not limited 
to the above event definitions or the above format for 
indicating which virtual entities are involved in a 
particular event. 

Once the event or events and the involved virtual 
entities are identified, the virtual reality system 20 
determines which particular communications function (s) to 
initiate (block 760) based on either current condition 
criteria (block 762), on configuration data or user 
profiles (block 764), or any combination thereof. If 
current conditions or configuration data indicate the 
need to change or modify the desired communications 
function (block 770) , the virtual reality system 20 
updates or modifies the desired communications 
function (s) (block 780) and then initiates the updated or 
modified communications function (block 790) . If no 
update or modification is required (block 770), the 
virtual reality system 20 simply initiates the configured 
communications function (block 790) . If the virtual 
reality environment 220 is to remain active (block 800) , 
the system returns to monitoring for virtual events 
(block 730), otherwise processing ends (820). 

Figure 8 provides additional details regarding a 
portion of the operating logic introduced in Figure 7 . 
Specifically, additional details appear for a method of 
implementing the process of registering entities (block 
720 of Figure 7). Registering begins (block 830) by 
defining a virtual entity, if necessary, and associating 
the virtual entity with a real world entity (e.g., 
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person, communication device or system) (block 840) . One 
or more events, possibly of different types, are defined 
for the selected virtual entity (block 850) . If the 
virtual reality system 20 is to support changeable 
communications functions (block 860) , the conditions and 
data bearing on the selection of a preferred 
communications function from one or more defined 
communications function are defined (block 870). The 
conditional evaluations to be applied to this information 
are then defined to establish the criteria that will be 
used, at a particular time, to select a preferred 
communications function (block 880) . Once this is done, 
or if there are no conditional evaluation criteria to be 
established (block 860) , the association between virtual 
events and the desired communications function is 
established (block 890) . 

This process may be thought of as "mapping" a set of 
defined virtual events into a set of desired 
communications functions. As noted, this mapping may be 
one-to-many, as in situations where one virtual event is 
associated with the initiation of more than one 
communications function. Alternatively, the mapping may 
be many-to-one, wherein multiple virtual events are 
associated with a common communications function. 
Further mapping subtlety arises from the ability to have 
the virtual reality system 20 dynamically change or 
update the mapping from virtual events to desired 
communications functions . 

Once the event-to-function mapping is completed for 
given events and a given entity, the virtual reality 
system 20 may return (block 900) to defining and 
associating more events and entities (block 840) . If 
there are no more entities to define and associate (block 
900) , processing ends (block 910) . 
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The foregoing descriptions and supporting drawings 
illustrate exemplary embodiments for practicing the 
present invention. However, as will be readily 
appreciated by those skilled in the art, the present 
5 invention allows significant variation in both 
implementation and operation. For example, the 
conditional criteria, if used, to determine which of 
several possible communications functions are to be 
initiated upon occurrence of a given event may be readily 

10 changed or expanded. Further, the manner of interfacing 
the virtual reality system 20 to one or more 
communications systems 30 may be changed as needed. For 
example, the virtual reality system 20 or parts thereof 
may be implemented as part of a computer network that 

15 hosts additional functionality, such as application and 
data support for a number of system users . In this 
circumstance, the computer network may include a hardware 
or software interface to the PSTN, a local PBX, the 
Internet, or any number of other communications systems. 

20 In such environments, then, the interface between the 
virtual reality system 20 and any supporting 
communications system 30 is intrinsic. 

No limitations should be construed for the present 
invention in light of the foregoing discussion of its 

25 exemplary embodiments. Indeed, the present invention is 
limited only by the scope of the claims appended thereto, 
and the reasonable equivalents thereof. 



